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Data and comment are presented on the methodologies of four recent studies 
which assessed school performance in cities experiencing unrest and conflict 
(Cincinnati, Columbus, Detroit, and Washington. D.C.). The topics of primary concern are 
0 ) the school performance objectives that were assessed, (2) the methods and tools 
used by the investigators of the dty systems in their assessment, (3) the conclusions 
reached about school performance and the recommendations offered for effecting 
improvements, and (4) specific recommendations made on how dty school systems 
might develop capabilities for continuous self-assessment. The four school systems 
studied relied heavily on standardized tet>:s to assess school performance. One of 
the major problems concerned the release of the test results to the public It is 
concluded that, in spite of remediation programs, students from poor families are still 
likely to achieve poorly. <HW) . 
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The focus of this paper is on methodology for studying one component of 



big city school systems — "assessment of school performance. " "How well 
are the schools doing?" This question is one that is raised repeatedly by parents 
and other interested, often anxious, citizens. The question is raised honestly 
and humbly, but generally with little knowledge of the complexities of answering 
the question — complexities due to the diversified nature of the institution and 
of the school enterprise; complexities because of ambiguity about school ob- 
jectives; complexities about social and intra-system norms against which assess- 
ment is to be made; and complexities because school officials and other pro- 
fessionals usually don't have an adequate base of information with which to 
answer the question. 

The data presented here were drawn from recent studies of four big cities: 

o 3.4 

Cincinnati, Ohio*; Columbus, Ohio 2 ; Detroit, Michigan ; and Washington, D.C. 

Primarily, questions dealt with in this paper are: (1) What school system 
objectives were delineated for purposes of assessing performance? (2) What 
did investigators of the city systems look for, and what tools did they use to 
assess school performance? (3) What conclusions were reached about school 



* This paper was prepared for a symposium "The Study of Big City School 
Systems: A Comparative Analysis" at the February, 1969 Convention of the American 
Educational Research Association, Los Angeles, California. 



performance, and what recommendations were offered for effecting improvements? 
And (4) What specific recommendations were made about how city school systems 
might develop capabilities for continuous self-assessment? 

SCHOOL PERFORMANCE OBJECTIVES 

Rarely are school studies called for except in a context of unrest and con- 
flict. In Cincinnati, the study followed failure of two public school referenda; 
Columbus citizens were anxious about intra-system equality of educational 
opportunity, and what they perceived to be less than adequate responsiveness 
of school officials to their questions; the Detroit study followed student refusal 
to attend classes, protesting what they considered to be inferior educational 
opportunity and less than adequate response of school administrators; and the 
"track system" and public dissatisfaction with the academic performance of 

school children preceded the school study in Washington, D.C. 

* * 

In such a climate of unrest, professionals associated with school studies 
are torn between their desire to conduct a study of scientific rigor and respect- 
ability, and one which has built-in capabilities for effecting rapid change. 

Both, I think, are possible and essential. To ignore the potential for gathering 
data for making generalizable recommendations is less that optimum professional 
responsibility; yet to ignore social climate in any school study is to overlook 
a set of crucial variables . 

When citizens ask about how well the schools are doing, what do they 
mean? It is doubtful that they have in mind the full range of school system 
performance measures which would assess input in terms of fiscal and human 



resources , assess output in terms of a more satisfied and productive society 
or accelerated economy, and which would evaluate all components of the 

* m 

teaching-learning processes that transpire between input and output. Usually 
they mean, "How well are students achieving academically in relation to ' 
achievements of other students in the same classroom in another school, through- 
out the system, or throughout the Country." 

f , 

In this paper, I have confined my discussion to pupil performance . The 
temptation is great in a school study to focus on a wide range of organizational, 
financial, personnel, instructional, and contextual variables that are essen- 
tially treatments, but which influence pupil performance. All of these are 
vital to the extent that they can be restructured or manipulated to affect improved 
performance of pupils . 

Performance objectives are not always clearly stated, nor is there wide- 
spread agreement about what they should be . Study teams that pre-determine 
performance objectives frequently find that the system being examined may not 
- have explicated its objectives , or its objectives may be different or be ordered 
differently from those of the study team. A more likely problem is the absense 
of data with which to assess important objectives. 

One task force of the Columbus Study team addressed itself to seeking 
answers to the questions, "How well are Columbus young people prepared for 
further schooling, employment, and community life?" This question, pertaining 
. only to pupii preparation, was one of four major questions posed. The Washington, 
D . C . team noted that the long range goal is not simply one of bringing children 
up to grade-level in reading; but, quoting John H. Fischer, "... to provide 
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schools that will assure every child an equal chance to use his talents to live 
effectively and to compete on fair terms in an open society." There are impor- 
tant educational objectives other than reading and mathematics achievement, 
the Washington team observed, but the schools are found with little data. These 
objectives are self-concept, ego-development, values, attitudes, aspirations, 
and other "non-academic", but important aspects of individual growth. Also, 
they noted, little is known about how schools are helping youngsters learn how 
to live and earn a living in a city. 

The Detroit team cautioned that citizens will no longer accept the fact 
that a school is at or near city-wide averages on standardized achievements 
tests, particularly at a time when city- wide averages are below national levels. 
And the Cincinnati team concluded that the most serious problem facing the 
Cincinnati School System at this time is the presence of large numbers of children 
who are achieving at standards well below those which are necessary for success 
in later schooling or in vocational life. 

Academic achievement as an important school objective cannot be denied. 
Parents have a right to expect professional expertise from the schools, and 
they have a right to demand positive results. The right to read, although 
not listed in "The Bill of Rights , " is among the most important of guaranteed 
rights to citizens. Beyond academic achievement, other objectives of the 
schools are not as clear. John Gardner once observed that education is "the 
servant of all our purposes. " Beyond that statement, there is little left. The 
great test of schools today is their capacity to deliver for those whose social and 
economic station in life leave them no alternative but to depend on the schools 



for equality of access to opportunities open to other citizens , but closed to 
them. There is nothing productive to be gained by protesting the severity 
of that burden which has been placed on the schools. The burden is still there , 
and problems of coping with the challenges must be faced directly, and shared 
openly and freely with all citizens . 

METHODS AND TOOLS 
FOR 

ASSESSING PERFORMANCE 

The methods and tools employed by study teams frequently tell more about 

performance objectives that are deemed important than do statements of objectives. 

5 

Restraints on Study process are always present, the most common one being time ; 

others are fiscal resources, manpower, and judgments about what information is 

fi 

most critical at the moment. 

COLUMBUS 

The Columbus team analysed all available scores of tests , administered 
in 1967-68. These included reading readiness for grade one; I.Q. , reading 
achievement (word meaning and paragraph meaning) , and arithmetic achievement 
for grade six; I.Q. , reading achievement and arithmetic achievement for grade 
eight; and reading achievement for grade nine . 

Each set of test scores for each grade level was graphed to show variance 
among six categories of socio-economic status. (In the report these SES 
categories are identified as priority one, priority two, etc.) A graph was pre- 
pared to report by SES category, reading achievement at grade one, grade six 
and grade nine. The purpose here was to depict the extent to which reading 



skills were sustained across the grades for each SES priority level. 

To assess factors other than achievement, the study team collected data 

from some 12,000 students to ascertain their perceptions, attitudes , and morale 

with respect to their school experiences. These included an open-end question- 
\ 

naire whereby high school seniors responded to the following topics: (1) going to 
school with children of different races; (2) going'to another school (in the suburbs 
or in the inner city) for all or part of my studies; (3) the counselors here; 

(4) feelings of people in my neighborhood about this school and the kind of 
education you get here. Morale with regard to a range of school experiences 
was tested in grades four, six, eight and ten. For grades nine and twelve, 
there were measures of authoritarianism, educational alienation, fatalism, 
general achievement motivation, self-concept, school achievement, motivation) 
and social desirability. All of the above measures were compared across SES 
categories. 

The team interviewed school drop-outs and other unemployed youths to 
get their views about their own school experiences, and the team studied 
performance of Columbus high school graduates during their freshman year at 
college. Also, data were collected from employers about how well the system's 
products were equipped for work. 

CINCINNATI 

The Cincinnati team analysed achievement test scores in reading, language, 
and arithmetic for grades two, four, five and eight, and they reported grade- 
level equivalencies for the first quartile, the median, and the third quartile for 

each grade level. 
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One comparison was made of the sixth grade I.Q. mean score in 1957 
with the sixth grade mean score in 1967. 

Enrollment in each of the three "tracks" for grades 7-12 was reported, 
noting disproportionate enrollment by race in each of the tracks. 

The team analysed and reported test results from evaluative studies of 
compensatory education in target elementary schools . 

The school holding power between grade seven and grade twelve (per- 
centage of grade seven enrollment who enrolled for grade twelve, five years 
later) was studied and reported. 

DETROIT 

The Detroit study differed from the others in that the team's mission was 
to look only at the high schools, whereas the other three studies were of all 
grade levels . 

vThe Detroit team analysed for each high school, scores of scholastic 
aptitude (SCAT) and educational progress (STEP); and the number of STANINES 
in the lower 25 per cent and in the upper 25 per cent were reported for each 
high school. The team arranged to have the Cooperative English Test admin- 
istered to all tenth and twelfth grade students in the English classes. The pur- 
pose of that study was to ascertain the extent of ability grouping across classes. 
Stated school policies were to group within classes rather than across classes; 
however, the team suspected that the latter was the case, operationally. 

The team administered an instrument to assess the extent of alienation 
in the student body. Alienation scores were reported in relation to effects on 
the scores of several variables: race, sex, socio-economic status, grade level, 
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and grade point average. 

Other matters related to school performance which the Detroit team studied 
were (1) attendance patterns, (2) tardiness patterns, (3) school holding effective- 
ness between grades ten and twelve, (4) school policies on grading, (5) school 

V * 

policies on discipline, (6) rate of student failure by school and by subject, 

(7) honors and scholarships, and (8) performance of graduates in college. 



WASHINGTON , D, C, 

The Washington, D.C. team conducted a study of pupil performance on 
standardized tests in a sample of 47 schools. 

The SCAT and STEP (reading and mathematics) scores for grade eleven 
students in eleven high schools were reported along with number of pupils , per 
cent of Negro pupils, number of teachers, and per- cent of Negro teachers in each 
of the schools. A similar set of data was reported for eleventh graders in eleven 
junior high schools. For sixth graders in 25 schools, these same data were 
reported along with three SES characteristics of each neighborhood. 

The percentage of five sample Washington groups who scored at or above 
national percentiles on the STEP reading, STEP mathematics, and SCAT test 
was reported for each of the following: fourth grade , sixth grade , ninth grade 
(general track) , eleventh grade (regular track) and eleventh grade (vocational 
high school) . 

Another pilot study of second graders in. three schools was conducted by 
the team to see if the school system had been successful in identifying students 
who were in need of special educational programs . 

Other relevant factors studied and reported by the Washington team were: 
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(1) post high school activities of the 1966 graduates, (2) location and types 
of higher education institutions attended by the 1966 graduates , 

(3) employers and types of employment of 1966 graduates and (4) patterns of 
high school holding power. 

« 

* i 

IMPROVING SCHOOL PERFORMANCE 

The four studies of big city school systems relied heavily on standardized 
achievements tests to assess school performance. The appropriate usage of 
tests is a metter of wide-spread concern. School officials have been 
indecisive about their testing policies # and they have guarded against 
release of scores to ’’unauthorized" persons. If professionals are confused 
about tests, so are parents and other citizens. Recently, there have been 
heated demands from the public for release of test scores . School systems 
have been slow to respond for at least two reasons: (1) they fear public 
misinterpretation of variance in scores across socio-economic sectors of 
the city, and (2) they frequently do not have scores compiled in any 
systematic way suitable tor public release. Citizens interpret this 
reluctance to mean that the scores are low and school officials are 
"covering up" inadequacies of the school system. 

The Columbus and Cincinnati study teams recommended that achieve- 
ment scores be released to the public. In both cities citizens were asking 

* 4T 

for public release of test scores . Public support of the schools is unlikely 
in. a context of suspicion that the schools are performing inadequately. 

The public's "right to know" is an educational principle that needs 
revitalizing. Parents are becoming more sophisticated about strengths and 
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weaknesses of tests; and a common concern around which the school and 

community might work together is improving school performance . The 

Cincinnati team stated the following with regard to test scores: 

These data (test scores) are useful in bringing 
about a better public understanding. of the nature 
and magnitude of the task of providing effective 
and efficient education in Cincinnati. They do 
not constitute an evaluation of the School system / 
and anv attempt to use them for this purpose would 
be an over-s implif ication of the problem . These 
data should be a matter of public record, because 
public support of education cannot be expected if 
the school system does not share with its public 
the nature and magnitude of the problems it faces . " 

In most of the studies achievement scores are reported in relation to 
SES variables; and in some cases, they are reported in relation to race. 
Their findings reflected lower achievement levels among the economically 
poor students than was the case for the more affluent students , with 

variance as high as 3. 8 grade levels . Further they noted that black 

\ 

students usually, score lower than white students because of overlap 
of SES and race variables . 

I suppose the other study teams went through the same dialogue as 
did the Columbus team. "Why study the obvious ? " There are mountains 
of achievement date that reflect the same patterns . That team cautioned 
itself about public credibility of the study unless local data were 
collected and reported. Their apprehensions were well founded. The 
Columbus community study reflected that 43 per cent of the citizens 
did not believe that there were special problems in educating inner city 



children. 



All study teams are, no doubt, perplexed about the extent to which 
the team itself should conduct studies of school performance, and the 
extent that it should analyse and report data collected by the school 
system. Time and resource constraints usually limit extensive studies 
by the team members beyond those which are deemed essential. On the 
other hand, few school systems have the research and evaluation 
capability for furnishing needed information. In a context of public 
unrest, there is also the problem of confidence' in locally supplied 
data. 

On questions regarding student morale, attitudes, perceptions, 
and the like, study teams have little alternative but to collect the 
data themselves. Students must be assured that teachers and school 
officials will not have access to their individual responses . Two of the 
studies reported here — Columbus and Detroit — did conduct studies of 
student morale which included measures of alienation and fatalism . 

In viev/ of Coleman's conclusion that the variable most highly 
associated with achievement is the extent to which a student feels he 
has control over his own destiny, a measure of this and related variables 
become especially significant.^ 

Unless school systems maintain follow-up -data on their graduates 
and dropouts, study teams are hampered in assessing how well the 
products of the schools are doing. The Washington team had access to 
school, system reports on employment patterns of its graduates for one 
year. The Columbus team had no such data, and the team relied on 



interviews and questionnaire, res ponses of local employers about how 
well the students were equipped for employment. Studies of how 
well graduates do in college are usually not done by school systems 
in .any systematic way. The Washington team reported on performance 
of students from the district at Howard University; the Detroit team 
studies performance of local graduates at Wayne State University; and 
• the Columbus team studies performance of local students at Ohio State 
University. These studies are restrictive, in that performance at 
local universities is difficult to generalize for likely performance 
at other universities . The reasons which prompt a student to go to a 
home -town college or university (such as part-time employment or low 
motivation) may mitigate against optimal performance . 

SCHOOL SYSTEM CAPABILITIES 
FOR SELF-ASSESSMENT 

City school system officials have become so burdened with day-to- 
day operations of the schools, problems frequently reach the crisis 
stage before they can.be handled. A capacity for planning,- research 
and evaluation to perceive potential problems before they arise, and 
to "service" the system for efficiency and effectiveness, is fondly 
hoped for, but rarely realized in school systems. When the problems 

mount beyond some undefined level of tolerance , an external agency is 

* 

frequently called for advice. Possibly the Dest advice from the agency 
beyond dealing with immediate crises would be to recommend ways 
whereby the school system can develop within itself la effective 



research and evaluation capability for planning and for operational 
decision-making . 

All of the four studies cited here called attention to this need. 

The Detroit team expressed concern that tests were being used for 
negative purposes of ascertaining deficiencies for decisions on 
grouping, rather than for diagnoses to plan instructional programs. 

The Columbus and Cincinnati teams recommended generous support 
of a division in the central office for research and evaluation. The 
division should be geared to (1) provide information to the public 
on how education dollars are spent, (2) provide quality indices on 
returns from educational investments, (3) evaluate how the school 
enterprise is performing, (4) generate sound data for operational 
decision-making at all levels , and (5) train teachers and others in 
ways to utilize information. ' 

The Washington team, likewise, recommended the establishment of a 
research and evaluation department which would encompass an existing 
group measurement division with a new mandate for diagnosis and assess 
ment primarily for instructional and counseling purposes. Also, the team 
called for a new model of psychological services which would stress 
educational, rather than clinical diagnoses. 

The Columbus team recommended the establishment of "Regional 
School Assessment Committees" composed of 12 members (three teachers, 
two students, one principal and six community leaders) . Their function 
would include review of achievement and other test data, react to 
curriculum change proposals, cousel the schools on community resources 



mediate community school grievances , evaluate disciplinary practices , 
and related matters * They would meet monthly and report at least 
annually to the Board of Education, the community, and to building level 
groups . 

Active participation of the community in assessing school performance 
and an effective research and evaluation program within the system may 

be essential for sustaining (in some cases restoring) public and 

« * 

professional confidence in the quality level of the schools . 

A REFLECTION ON SCHOOL PERFORMANCE ASSESSMENTS 
A final reflection on studies of school performance leaves me 
with the disturbing conclusion that in spite of remediation programs, 

students from poor families are still likely to achieve poorly. To 

explain away unsatisfactory achievement because of environmental 

factors beyond the control of schools is as immobilizing as explaining 

it away because of inherited physiological characteristics . Schools 

can and must .give leadership to effecting racial and socio-economic 

heterogeneity. In the meantime there is a job to be done. 

Superintendent Carl Dolce reflects fairly accurately my sentiments in 

his discussions of the school as a delivery system that must produce. 

His questions are hard, but they require answers: 

What is happening to children? Not much! We have 
thrown out the crutches that educators have used 
for a long period of time. One of the crutches is 
large class size. Okay, we say, we will reduce 
class size, and then we are told, *Well we don't 
have enough supplies and equipment. ' So we pump 
in supplies and equipment, and now what we are . 
getting is , 'Well, the problem is the community. ' 



Where does the rationalization stop? And where, 

and at what point, are people expected to produce? 10 

« 

What happens to students when recommendations of big city 
study teams are implemented is the one pervasive challenge that 
preempts all others , and one that frequently haunts me . 
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